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METHOD AND SYSTEM FOR MAINTAINING FREEFORM INK ^ ft !^ L ^ 
ANNOTATIONS ON CHANGING VIEWS fO-0 

BACKGROUND OF THE INVENTION 

1. Field of Invention 

5 This application is directed to an electronic system for displaying changing 

views. In particular, the invention is directed to permitting freeform digital ink 
annotation on electronic media and to maintaining the logical relationship between 
freeform digital ink annotations and objects in changing views of the media. 

2. Description of Related Art 

10 Freeform ink annotation of paper documents is a wide-spread practice because 

it aids many aspects of document work, including the processes of thinking, 
categorizing and reviewing. In short, freeform ink annotation has proven to be a 
powerful comprehension tool. Paper and paper-like documents (such as PDF, 
postscript, image files, etc.) have a static layout and content. The static nature of 

1 5 these documents makes annotation simple. 

However, the kinds of documents that people use are increasingly created and 
presented digitally on displays using computers. Such documents are often dynamic 
and, therefore, the accurate representation of the relationship of the annotations to 
objects in the display has been impossible because of the changing display. For 

20 example, a display of a word processor document changes as the document is 

undergoing revisions. In a similar manner, information visualization systems such as 
the hyperbolic browser disclosed in A Focus + Context Technique Based on 
Hyperbolic Geometry for Visualizing Large Hierarchies* by Lamping et al., in 
Proceedings of ACM SIGCHI Conference on Human Factors and Computing 

25 Systems, May 1995, change the display of the document as the user navigates the 
information space. 

A freeform digital ink whiteboard known as Tivoli is disclosed in Pen-Based 
Interaction Techniques For Organizing Material on an Electronic Whiteboard, by 
Moran, et al., in UIST '97, ACM press. Tivoli allows users to identify a connection 
30 between two areas in a display, where that connection is maintained as those areas are 
moved and the display changes. Circle marks which surround other ink identify areas, 
and freeform digital ink marks which connect two area boundaries together form a 



connection between those two areas. This connection persists as the areas move and 
Tivoli attempts to maintain the shape of the connecting mark. 

To assist in navigation of documents conventional systems use a set type of 
bookmark. Examples of such navigation systems are disclosed in Virtual Annotation 
5 System, by Harmon, et al., in Proceedings of the IEEE 1 996 Virtual Reality Annual 
International Symposium, pp. 239-245 (1996) and in Worldlets:3D Thumbnails for 
Wavfindinp in Virtual Environments, by Elvins, et al., in video proceedings of UIST 
'97, pp. 21-30 (1997). These systems make it ackward for a user to freely interact 
with the navigation system because the user is required to specify an annotation 

10 command before commenting. 

Current systems attempt to manage freeform digital ink annotation on 
changing views or displays by at least one of three methods (the terms "view" and 
"display" are used interchangeably throughout this application). The first method, 
known as the static approach, treats ink annotations as associated with a single view. 

1 5 When the presentation or the content of the document changes, the annotations either 
disappear or remain in their original positions. A system known as "MATE" which is 
disclosed in A Marking Based Interface for Collaborative Writing, by Hardock et al., 
in Proceedings of UIST '93 (1993), addresses changing content by maintaining a static 
annotated version, a changed version without annotation and logical links between 

20 these two versions. 

A second method, known as the snapshot approach, treats the dynamic view as 
a series of static snapshots which can be individually annotated. Examples of this 
approach include the Wang Freestyle disclosed in Rapid, Integrated Design of a 
Multimedia Communication System, by Francik, in Human-Computer Interface 

25 Design, Morgan Kaufman Publishers, Inc., (1 996). 

The third method, known as the temporal association approach, separates the 
ink annotations from the view and links the two temporally, so that marks made in a 
specific time are associated with the view at that time. Examples of this approach are 
used in Marquee which is disclosed in Marquee: A Tool for Real-Time Video 

30 Logging, by Weber et al, in Proceedings of CHI '94, ACM Press, (April 1994) and 
Dynomite which is disclosed in Dvnomite: A Dynamically Organized Ink and Audio 
Notebook, by Schilit et al., In Proceedings of CHI *97, ACM Press, (1997). 



SUMMARY OF THE INVENTION 
In conventional interfaces that accept freeform digital ink annotation, the 
indication of the logical relationships between the annotations and the associated 
objects are not maintained if the view changes. For example, if a user annotates the 
5 text of a document and, subsequently, the annotated text reflows across lines, the 
annotation does not follow the reflowed portion of the text. Therefore, while there 
may be an underlying logical relationship to the annotated text, the layout of the new 
view does not indicate this logical relationship between the text and the annotation. 

The systems described as related art above also do not maintain any indication 
10 of logical relationships between the annotations and the associated objects if the view 
changes. Tivoli does not maintain the freeform digital ink annotations as the areas 
move around and the display changes. Rather, Tivoli maintains only one specific type 
of annotation: a single stroke from the boundary of one area to another. This 
restriction precludes the use of freeform digital ink annotations in changing displays 
15 to label moving objects or to aid navigation. 

While MATE maintains the logical relationships between freeform digital ink 
annotations and the corresponding underlying text, it does not maintain the indications 
of these logical relationships as the view changes. 

Freestyle does not allow the freeform digital ink annotations to carry over to 
20 new views of the same document 

Dynomite does not permit annotations directly on the view and the annotations 
are not rendered on the new views in accordance with changes to the views. 

The method and system of the invention maintain the layout relationships 
between freeform digital ink annotations and objects in changing views. As a user 
25 makes a freeform digital ink annotation on a view and as the view changes, the 

method and system of the invention adapt the marks in accordance with the changes. 
Thus, the logical relationship between the freeform digital ink annotations and the 
objects made in one view can be understood in future views based upon the layout 
relationships. 

30 The method and system also maintain the indication of the logical 

relationships between freeform digital ink annotations and objects in dynamic media 
such as video and interactive virtual worlds. For example, if a user, who is exploring 
an information space with a hyperbolic browser, makes a freeform digital ink 



annotation on a node of the tree, when the user changes the display to bring another 
portion of the tree into focus, the freeform digital ink annotation rotates and scales 
along with the tree to continue to indicate the logical relationship of the freeform 
digital ink annotation to the node. 
5 The methods and systems of this invention maintain the indication of the 

logical relationships between freeform digital ink annotations and objects in a view as 
the view of the annotated objects changes. This invention brings the experience of 
marking on paper to dynamic media, changing word-processor documents, 
information visualizations, virtual worlds, digital video and word processors. 
10 The system and method of the present invention maintain the indication of the 

logical relationship, even if the objects that are logically related to the freeform digital 
ink annotations are deleted, inserted, copied or moved or otherwise altered. The 
method and system also maintain the indication of the logical relationship if the layout 
presentation of the objects in the display change because the objects reflow, scale, 
1 5 resize, rotate, stretch, distort or morph or if any other aspect of the layout presentation 
changes, regardless of whether these changes occur discretely or continuously. 

Additionally, another exemplary embodiment of the systems and methods of 
the invention provides navigational markers that make freeform digital ink annotation 
a powerful tool for navigating dynamic documents. A user is permitted to select a 
20 freeform digital ink annotation in an arbitrary view, and the display is returned to the 
state it was in when the selected freeform digital ink annotation was made. 

BRIEF DESCRIPTION OF THE DRAWINGS 
The preferred embodiments of this invention will be described in detail, with 
reference to the following figures, wherein: 
25 Fig. 1 is a block diagram of an embodiment of the system for displaying 

changing views of this invention; 

Fig. 2A shows a display of an annotated hyperbolic browser in accordance 
with an embodiment of the present invention; 

Fig. 2B shows the annotated hyperbolic browser of Fig. 2A with the view 
30 changed and freeform digital ink annotations adjusted in accordance with the change 
in the view in accordance with an embodiment of the invention; 
Fig. 3A shows a display of an annotated document; 



Fig. 3B shows the annotated document of Fig. 3A after modification of the 
document and shows the adjustments to the freeform digital ink annotations in 
accordance with an embodiment of the present invention; 

Figs. 4A-4F show a flowchart outlining a control routine for mapping freeform 
5 digital ink annotations to objects in a display in accordance with an embodiment of 
the prevent invention; 

Figs. 5A-5B show a flowchart outlining a control routine for rendering 
freeform digital ink annotations on a changed view in accordance with an embodiment 
of the present invention; and 
10 Fig 6 shows a few examples of different types of freeform digital ink 

annotations. 

These and other features and advantages of this invention are described in or 
are apparent from the following detailed description of the preferred embodiments. 
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 

15 Fig. 1 shows one exemplary embodiment of the electronic system 10 of this 

invention. The electronic system 10 includes a processor 12 communicating with a 
memory 14 that stores the programs, documents and other data for practicing the 
invention. The processor 12 also communicates with a display 16, a keyboard 18, a 
mouse 20 and a pen 22 via input/output device 24. The keyboard 1 8, mouse 20, pen 

20 22 and any other interface devices (not shown) are operated by a user to control the 
operation of the electronic system 10. Control of the electronic system 10 is not 
limited to the interface devices shown and does not require any particular interface 
device. The display 16 may be on a stand-alone pen computer or a tablet connected 
by a tether to a conventional computer or any other type of display. 

25 As shown in Fig. 1 , the system 10 is preferably implemented using a 

programmed general purpose computer. However, the system 10 can also be 
implemented using a special purpose computer, a programmed microprocessor or 
microcontroller and any necessary peripheral integrated circuit elements, an ASIC or 
other integrated circuit, a hardwired electronic or logic circuit such as a discrete 

30 element circuit, a programmable logic device such as a PLD, PLA, FPGA or PAL, or 
the like. In general, any device on which a finite state machine capable of 
implementing the flow charts shown in Figs. 4A-5B can be used to implement the 
system 10. 



Additionally, as shown in Fig. 1, the memory 14 is preferably implemented 
using static or dynamic RAM. However, the memory 14 can also be implemented 
using a floppy disk and disk drive, a writable optical disk and disk drive, a hard drive, 
a flash memory or the like. Additionally, it should be appreciated that the memory 14 
5 can be either distinct portions of a single memory or physically distinct memories. 

Further, it should be appreciated that the links 26, 28, 30, 32, 34 and 36 
connecting the memory 14, the display 16, the keyboard 18, the mouse 20, the pen 22 
and the input/output device 24 to the processor 12 can be wired or wireless links to 
networks (not shown). These networks can be local area networks, wide area 
10 networks, intranets, the Internet, or any other distributed processing and storage 
networks. 

Fig. 2A shows a display of a view of a hyperbolic browser 40. The display 40 
includes a freeform digital ink annotation 42. Fig. 2B shows an alternative view of 
the document shown in Fig. 2A. The freeform digital ink annotation 42 has been 

15 scaled and moved so that the layout presentation of the freeform digital ink annotation 
in the view corresponds with the change of the view of the objects in the document 40 
to maintain the indication of the logical relationship between the freeform digital ink 
annotation 42 and the objects. 

Fig. 3 A shows a first view 50 of a document 52 having freeform digital ink 

20 annotations 54, 56 and 58. Fig. 3B shows a second view 60 ofthe document 52 after 
the document has been revised. Fig. 3B shows that the freeform digital ink 
annotations 54, 56 and 58 are adjusted in accordance with the logical relationships to 
the objects in the document 52 in accordance with an embodiment of the invention. 
The content ofthe document 52 shown in Fig. 3 A has been edited to insert the text "is 

25 so much fun to do" at 62 and to delete approximately two full lines of text at 64. In 
response to these edits, the text that has been annotated at 54 now flows across two 
lines and the "cross-out" freeform digital ink annotation 54 flows along with the 
underlying text objects to continue to indicate the logical relationship between "cross- 
out" freeform digital ink annotation and the underlying text objects. The invention 

30 also recognizes that the group of ink strokes at 58 is a margin note and the system 

slightly adjusts the vertical position of the margin note 58 in the margin to continue to 
indicate the logical relationship between the margin note 58 and the reflowed line. 
Additionally, the freeform digital ink annotation 56 is adjusted to adapt to the one line 



being removed between the anchor points at 66 and 68 to continue to indicate its 
logical relationship to the text. 

An embodiment of the method and system of the present invention involves at 
least a three-step process. The first step is adding the freeform digital ink annotation 
5 on objects in a first view. The second step involves mapping segments of the 

freeform digital ink annotations to corresponding objects in the view. The third step 
involves moving, scaling or otherwise transforming the freeform digital ink 
annotation to maintain the indication of the logical relationship between the freeform 
digital ink annotation and the corresponding objects as the display changes. 
10 Optionally, in an embodiment of the invention the freeform digital ink 

annotations can be used as navigational markers. For example, the user can select an 
annotation in a view and return to the annotated view that existed when the annotation 
was created. 

The freeform digital ink annotation of objects in changing views of an 
15 embodiment of the invention relies on an object model for the view. In the example 
of Figs. 2A and 2B, the objects are the nodes in the hyperbolic browser. In the 
example of Figs. 3 A and 3B, the objects are the text characters. In a third example 
(not shown), the objects are the output from an object tracking system running on a 
video data stream. 

20 An embodiment of the method and system of the invention maps the freeform 

digital ink annotations to objects in a view by placing anchors in the underlying 
document. For example, a simple circling, highlighting or a crossing out mark is 
anchored to the corresponding span of character positions that are covered by the 
freeform digital ink annotation in the view. In the example shown in Figs. 3A and 3B, 

25 a comment 58 in the margin is anchored to a position in the document in the margin 
near all of the corresponding words or characters. 

More complex freeform digital ink annotations may have several anchors. For 
example, as shown in Figs. 3A and 3B, consider a freeform digital ink annotation 56 
that includes a circle 68 connected to an arrow 66. The circle 68 is anchored to the 

30 span of words or characters that is selected by the circle and the tip of the arrow 66 is 
anchored to its end point while the line connecting the circle 68 and the tip of the 
arrow 66 is not anchored. 



An even more complex freeform digital ink annotation may involve logical 
relationships to moving objects in a virtual world. An example of such an annotation 
involves the freeform digital ink annotation of a door to a room in a virtual world. 
The layout of the door changes in a view of the virtual world as the door is opened or 
5 closed. The method and system of an embodiment of the invention maintain the 

indication of the logical relationship between the freeform digital ink annotation and 
the door as the view of the door changes. Thus, the user sees the freeform digital ink 
annotation rotating with the door. 

For the purposes of the following discussion, the following definitions are 

10 provided: A position is a two-dimensional coordinate in a view. A stroke is a list of 
positions in a view. An offset is a two-dimensional offset from a position in a view. 
A region is a portion of a two dimensional view that is represented, for example, by a 
bounding box or by a list of positions. A character is a character in a text document. 
Character positions between characters in the flow of text are maintained as the text 

1 5 changes. A margin anchor is a character position and a boolean flag for left or right 
margins. A simple anchor is a character, a character position or a margin anchor. A 
composite anchor is a pair of simple anchors with offsets. An anchor is either a 
simple anchor or a composite anchor. An anchored stroke is an anchor plus a list of 
offsets. 

20 An embodiment of the method and system of the invention moves the anchor 

points if the underlying anchored objects move. Unanchored segments of the 
freeform digital ink will stretch to fit between the anchored points. Sometimes 
adjacent anchor points move apart because the underlying objects move apart: For 
example, if a text phrase reflows from a single line across two lines (as shown in Figs. 

25 3A and 3B). In such cases, the connection between the formerly adjacent anchored 
points can be left out to reduce clutter and any unanchored ink between those points 
can be removed. Thus, as shown in Fig. 3B, the reflowed freeform digital ink 
annotation is split and does not show ink extending between the two segments, which 
are on two separate text lines. If anchor objects for a stroke are deleted, then the 

30 freeform digital ink annotations can be remapped to the nearest remaining objects in 
the document If all anchor objects for the freeform digital ink annotation are deleted 
then the annotation can be deleted. 



The type of the anchors for the freeform digital ink annotation depends upon 
the shape of the annotation and the kind and number of the corresponding objects. 
One method, detailed below, for computing ink anchors is to select the nearest object 
for each segment in the ink. Alternatively, the type of stroke, as determined from its 
5 shape, can be used to determine the anchor objects. In another case, a freeform digital 
ink annotation with multiple anchors may cause unwanted distortions when the view 
changes. To avoid this problem, an embodiment of the present invention groups 
segments of the freeform digital ink annotation together to reduce the number of 
anchors. 

10 As briefly described earlier, an embodiment of the method and system of the 

present invention uses at least three processes. The first process receives freeform 
digital ink annotations on the view, the second process computes a logical mapping 
between objects in the view and the freeform digital ink annotations, and the third 
process renders the freeform digital ink annotations on a view that has changed. An 

1 5 optional fourth process selects freeform digital ink annotations to navigate to a 
previous state. 

The first process includes sensing data points from a pointing device, such as a 
mouse or a pen, rendering the ink strokes by connecting the points on the view, and 
grouping the ink strokes together to form a freeform digital ink annotation. These ink 

20 strokes may be grouped, for example, by a simple time threshold. The second process 
receives the freeform digital ink annotations as input and maps between the objects 
and the freeform digital ink annotations. The third process updates the view when the 
document changes. This process uses the mapping to render the freeform digital ink 
annotation on the objects as the view changes. 

25 The procedures and the embodiments of the invention described herein assume 

that the objects do not change while the user is inking. However, the method and 
system of the invention is not limited to applications in which the objects being 
annotated do not change. As an example, in the case of video, the video may be 
halted while the user marks. Alternatively, the video can continue while the user 

30 marks and the state of the objects at some point in time can be used to interpret the 
objects and update the mapping between the objects and the freeform digital ink 
annotations. 



The fourth process takes a selection on a view, determines if it corresponds to 
a freeform digital ink annotation, and, if it does, replaces the current view with the 
view which existed when the freeform digital ink annotation was made. This process 
requires storing an additional mapping from the freeform digital ink annotations to the 
5 state of the changing view. 

The present invention recognizes that objects may be built from sub-objects 
which may be split from each other. One example is a text phrase that is built out of 
words. If a phrase has a freeform digital ink annotation that circles the phrase and if 
that phrase reflows across two lines, the invention recognizes this split and may either 

10 reflow the freeform digital ink annotation by splitting it up into several segments, 
each of which follows the corresponding underlying words as they move, or the 
system may choose to draw lines between matching segments of the circle. 
Alternatively, the freeform digital ink annotation could be replicated at a smaller scale 
on each individual word. 

15 A user can also copy a phrase to another paragraph, which could be considered 

a form of splitting or as an entirely new and unrelated object, without the circle 
annotation. 

Conversely, objects can merge into larger entries. For example, if an 
unannotated word between two annotated words is deleted then the two annotated 

20 words "merge" to form an annotated phrase. The two annotations join to form a 
single annotation of the merged phrase. 

Objects may also disappear entirely. For example, words in a text editor can 
be deleted. In this case, the freeform digital ink annotations associated with the object 
can disappear also, or they can remain at the location between the surrounding objects. 

25 Objects can also change. For example, a user in a text editor can select an 

annotated word and replace it with a new word. The new word could be considered a 
a new version of the original object. The annotation on the original word may then be 
associated with the new object. In this case, the annotation may alter its shape to 
conform to the new word, for example, by stretching or shrinking. Alternatively, the 

30 annotation could be removed from the display. 

An embodiment of the method and system of this invention may group ink 
strokes on text to form a single freeform digital ink annotation. By way of 
non-limiting examples, the annotations can be categorized as circles, 



underlines/cross-outs, insertions, margin notes, and connectors. Annotations that are 
circles are identified by ink stroke circles that contain text and these ink stroke circles 
are mapped to the text objects inside the ink stroke circle. Underline/cross-out 
annotations are identified by elongated marks which are either directly on the text or 
5 underneath the text. These annotations are mapped to the text objects that they are on 
or underneath. An annotation may be identified as an insertion if the annotations dips 
into a line of text. An insertion annotation maps to the position where the insertion 
mark appears in the text, for example, between two characters. A margin note is an 
annotation which is located in a margin. The margin note annotations are mapped to 

10 the text that is adjacent to the vertical and/or horizontal position of the annotations. 
Connector annotations are identified as connectors by arrows that m^y appear at the 
end of the annotation or by lassos (i.e., a circle plus a line and a single stroke). A 
connector annotation is split into a circle and a connector portion. Connector 
annotations are mapped to two objects independently. The first object is the object 

1 5 that occurs at the start of an annotation and the second object occurs near the end of 
the annotation. These objects may be positions in the text, or they may be other 
annotations. As an example, one end of the line portion of a lasso is mapped to the 
circle annotation which, in turn, maps to the text. Another example is a line from the 
text to a margin annotation. One end of the line is mapped to the text while the other 

20 end is mapped to the margin annotation. 

Fig. 6 shows several exemplary types of freefonn digital ink annotations. Fig. 
6 shows a display of a document 400 including circle (or enclosure) annotations 402 
and 404, an underline annotation 406, cross-out annotations 408 and 410, insertion 
annotations 412 and 414, a margin annotation 416, a line to margin annotation 418 

25 (with associated margin note 420) and an arrow connector 422. It should be 

understood that the types of freeform digital ink annotations shown in conjunction 
with the descriptions of various exemplary embodiments herein are intended to be 
illustrative and not limiting to any particular embodiment/type or grammar of 
annotation. 

30 The method and system of this invention is based upon the concept that 

anchors are determined for freeform digital ink annotations. Anchors are offsets from 
objects. Freeform digital ink annotations are mapped to the object structure of the 



view underlying the freeform digital ink annotations. While it is to be understood that 
the mapping may be to hierarchical object structures such as characters, phrases and 
paragraphs, for the sake of simplicity, the following description describes a mapping 
of a single level: between freeform digital ink annotations and characters or positions 
between characters. A text editor maintains these characters and positions as the text 
changes. For characters that have been deleted, the text editor marks the characters 
and their corresponding character positions as deleted, but the text editor still 
maintains a correspondence between the character positions and some location 
between remaining characters. 

An embodiment of the method and system of the invention may rely upon a 
text editor that stores the sequence and presentation of characters. The embodiment 
can determine if a character position, a character or a range of characters has been 
deleted; can acquire the position in the view that corresponds to a character position; 
acquire the region in the view that corresponds to a character; can obtain left and right 
margin positions that correspond to a character position; can acquire a range of 
character positions that correspond to a region in a view; and can split a freeform 
digital ink annotation into a set of substrokes or segments, such that each substroke 
corresponds to a single character or to a position between characters. 

One embodiment of the method and system of the invention may also use an 
annotation database that stores anchored strokes. The annotation database may be 
accessed to add an anchored stroke to the database, to delete an anchored stroke from 
the database or to acquire a list of all of the anchored strokes. 

Figs. 4A through 4F show a flow chart outlining a control routine for mapping 

freeform digital ink annotations to objects in an underlying view in accordance with 

an embodiment of the invention. The flow chart parallels the following pseudocode. 

First, classify the group of strokes as circles, underlines/cross-outs, insertions, 
margin notes, or connectors. Notice that this may require splitting some of the 
strokes, and may result in several groups of strokes. Then, 
, Sort the new groups of strokes so that connectors appear after any groups they 
connect to 

For each new group of Strokes 

If the group is a circle or an underline or a cross-out, then 

Split the group of strokes into a list of strokes and characters 
For each stroke and character pair in the list 

Create a new simple anchor corresponding to the 
character 

Get the region corresponding to the character 



Let anchor position be the center of the region 

Create a new anchored stroke based on the simple anchor 

For each position in the stroke, 

Let offset be position minus anchor position 
Append offset to the anchored stroke's list of offsets 

End for 

Add anchored stroke to the annotation database 
End for 

Else if the group is an ink insertion into the text, then 

Let character position be the character position where the ink dips 
into the text 

Get the position corresponding to the character position 
Let anchor position be that position 
For each stroke in the group 

Create a new anchored stroke based on the character 

position 

For each position in the stroke 

Let offset be position minus anchor position 
Append offset to the anchored stroke's list of offsets 

End for 

Add anchored stroke to the annotation database 
End for 

Else if the group is a margin annotation, then 

Let character position be the first character position corresponding 
to the vertical extent of the annotation 

Get the left and right margin positions corresponding to character 
position 

If the group is mostly to the left of the left margin position, then 
Let margin anchor be { character position, flag=left } 
Let anchor position be the left margin position 

Else 

Let margin anchor be { character position, flag=right } 
Let anchor position be the right margin position 

End if 

For each stroke in the group, 

Create a new anchored stroke based on the margin anchor 

For each position in the stroke 

Let offset be position minus anchor position 
Append offset to the anchored stroke's list of offsets 

End for 

Add anchored stroke to the annotation database 
End for 

Else if the group is a connector, then 

For each of starting and ending position on connector 
If position is near an annotation 

Let simple anchor be the anchor for that annotation 
Let anchor position be the current position of the 
simple anchor 

Let offset be position minus anchor position 



Else (position is near an object) 

Let simple anchor be the anchor for that object 
Let anchor position be the current position of the 
object 

Let offset be position minus anchor position 

End if 
End for 

Let composite anchor be the anchors and offsets corresponding to 
the starting and ending positions 
For each stroke in the group 

Create a new anchored stroke based on the composite 

anchor 

For each position in the stroke 

Let offset be position minus starting position 
Divide offset by the distance between thestarting 
and ending positions 

Rotate offset around the origin by-angle (ending 
position minus starting position) 
Append offset to the anchored stroke's list of offsets 
End for 

Add anchored stroke to the annotation database 
End for 

End if 
End for 

The control routine shown in Figs. 4A-4F starts at step SI 00 and continues to 
step SI 02. In step SI 02, the control routine groups the ink strokes into annotations 
such as circles, underlines/cross-outs, insertions, marginalia or connectors. It is to be 
appreciated that this may require splitting some strokes and may result in several 
groups of strokes. The control routine then continues to step SI 04 where the 
annotations are sorted so that connectors appear after any group to which they are 
connected. The control routine then continues to step SI 06 where the control routine 
initiates a loop that iteratively processes all of these annotations. The control routine 
determines if the current annotation is a circle, an underline or a cross-out. If in step 
SI 06, the control routine determines that the current annotation is a circle, an 
underline or a cross-out then the control routine jumps to step SI 16. If, on the other 
hand, in step SI 06 the control routine determines that the current annotation does not 
include a circle, an underline or a cross-out then the control routine continues to step 
S108. In step S108, the control routine determines if the current annotation is an ink 
insertion. If, in step SI 08, the control routine determines that the current annotation is 
an ink insertion then the control routine jumps to step SI 38. If, on the other hand, the 
control routine determines that the current annotation is not an ink insertion, then the 



controfroutine continues to step SI 10. In step SI 10, the control routine determines if 
the if the current annotation is a margin annotation. If, in step SI 10, the control 
routine determines that the current annotation is a margin annotation then the control 
routine jumps to step S158. If, on the other hand, in step SI 10, the control routine 
5 determines that the current annotation is not a margin annotation then the control 
routine jumps to step SI 86. 

After completing one of the processes of Figs. 4B-4F, which will be explained 
in detail below, the control routine continues to step SI 12 where the control routine 
determines if more annotations need to be processed. If in step SI 12, the control 
10 routine determines that more annotations need to be processed, then the control 

routine returns to step S 106. If, on the other hand, the control routine determines that 
no more annotations need to be processed, then the control routine continues to step 
SI 14 where control is returned to the environment in which the control routine of 
Figs. 4A-4F operate. 

15 At step SI 16, the control routine splits the annotation into a list of substrokes 

each with a corresponding single character. Thus, the control routine splits the 
annotation into substroke/character pairs. The control routine then continues to step 
SI 1 8, where the control routine initiates a loop to iteratively process each 
substroke/character pair in the current annotation. In step S 1 1 8, the control routine 

20 creates a new simple anchor that corresponds to the current character. The control 
routine then continues to step SI 20 where the control routine acquires the region in 
the view that corresponds to the current character. The control routine then continues 
to step SI 22 where the control routine sets an anchor position to the center of the 
region in the view. The control routine then continues to step SI 24 where the control 

25 routine creates a new anchored stroke that is based upon the simple anchor and 

continues to step S126. In step S126, the control routine sets an offset to the position 
from the substroke minus the anchor position and continues to step SI 28. In step 
S128, the control routine appends the offset to the anchored stroke's list of offsets and 
continues to step SI 30. In step SI 30, the control routine determines whether more 

30 positions exists in the current substroke. If the control routine determines in step 
S130 that more positions exists in the current substroke, then the control routine 
returns to step SI 26. If, on the other hand, the control routine determines that more 
positions do not exist in the current substroke then the control routine continues to 



step S132. In step SI 32, the control routine adds the anchored stroke to the 
annotation database, and the control routine then continues to step SI 34. In step 
SI 34, the control routine determines if more substroke/character pairs exist in the 
current annotation. If, in step SI 34, the control routine determines that more 
substroke/character pairs exist in the current annotation then the control routine 
returns to step SI 18. If, on the other hand, the control routine determines, in step 
SI 34, that no more substroke/character pairs exist in the current annotation then the 
control routine continues to step SI 36 where control returns to step SI 12. 

At step SI 38 the control routine sets a current character position to the 
character position where the annotation dips into the text. The control routine then 
continues to step SI 40 where the control routine acquires the position corresponding 
to the current character position. The control routine then continues to step S142 
where an anchor position is set to the current character position. The control routine 
then continues to step SI 44 where the control routine initiates a loop to process all 
strokes in the annotation. In step S144, the control routine creates a new anchored 
stroke based upon the current character position as an anchor and the control routine 
continues to step S146. In step S146, the control routine sets an offset to the current 
character position from the stroke minus the anchor position and continues to step 
S148. In step S148, the control routine appends the offset to the anchored stroke's list 
of offsets and continues to step SI 50. In step SI 50, the control routine determines 
whether more positions exists in the current stroke. If, in step SI 50, the control 
routine determines that more positions exist in the current stroke then the control 
routine returns to step S146. If, on the other hand, in step S150, the control routine 
determines that no more positions exist in the current stroke then the control routine 
continues to step SI 52. In step SI 52, the control routine adds the anchored stroke to 
the annotation database and the control routine continues to step SI 54. In step SI 54, 
the control routine determines whether more strokes exist in the current annotation. 
If, in step SI 54, the control routine determines that more strokes exist in the current 
annotation then the control routine returns to step SI 44. If, on the other hand, the 
control routine determines, in step SI 54, that no more strokes exists in the current 
annotation then the control routine continues to step SI 12. 

At step SI 58, the control routine sets a character position to the first character 
position corresponding to the vertical extent of the current annotation. The control 



routine then continues to step SI 60 where the text editor acquires the left and right 
margin positions that correspond to the character position and continues to step SI 62. 
In step SI 62, the control routine determines whether the annotation is mostly to the 
left of the left margin. If, in step S162, the control routine determines that the 
5 annotation is mostly to the left of the left margin, then the control routine continues to 
step SI 64. If, on the other hand, the control routine determines that the annotation is 
not mostly left of the left margin then the control routine jumps to step SI 68. In step 
S 164, the control routine sets the margin anchor to LEFT and the control routine 
continues to step SI 66. In step SI 66, the control routine sets an anchor position to the 

10 left margin position and continues to step S172. 

In step SI 68, the control routine sets a margin anchor to RIGHT and continues 
to step SI 70. In step SI 70, the control routine sets an anchor position to the right 
margin position and continues to step SI 72. In step SI 72, the control routine initiates 
a loop to process all strokes in the current annotation. In step SI 72, the control 

1 5 routine creates a new anchored stroke based upon the margin anchor and continues to 
step SI 74. In step SI 74, the control routine sets an offset to the position from the 
current stroke minus the anchor position and continues to step S 1 76. In step S 1 76, the 
control routine appends the offset to the anchored stroke's list of offsets and continues 
to step SI 78. In step SI 78, the? control routine determines whether more positions 

20 exist in the current stroke. If, in step S 1 78, the control routine determines that more 
positions exist in the current stroke then the control routine returns to steps S I 74. If, 
on the other hand, in step S 178, the control routine determines that no more positions 
exists in the current stroke then the control routine continues to steps SI 80. In step 
SI 80, the control routine adds the anchored stroke to the annotation database and 

25 continues to step S 1 82. In step S 1 82, the control routine determines whether more 
strokes exists in the current annotation. If, in step SI 82, the control routine 
determines that more strokes exists in the current annotation, then the control routine 
returns to step SI 72. If, on the other hand, the control routine determines, in step 
S 1 82, that no more strokes exists in the current annotation then the control routine 

30 continues to step S 1 56 where the control routine returns to step S 1 1 2. 

At step SI 86, the control routine initiates a loop to process all starting or 
ending positions in the current annotation. In step SI 86, the control routine 
determines whether the current of the starting and ending positions are near another 



annotation or an object. If in step SI 86, the control routine determines that the current 
position is near another annotation then the controller continues to step SI 88. If, on 
the other hand, the control routine determines, in step SI 86, that the position is not 
near another annotation then the control routine continues to step S192. In step SI 88, 
5 the control routine sets a simple anchor as the anchor for the current annotation and 

continues to step SI 90. In step SI 90, the control routine sets an anchor position as the 
current position of the simple anchor and continues to step SI 96. In step SI 92, the 
control routine lets a simple anchor be the anchor for an object which is near the 
current starting or ending position and the control routine continues to step SI 94. In 

1 0 step S 1 94, the control routine sets an anchor position to the current position of the 
object and the control routine continues to step SI 96. In step SI 96, the control 
routine sets an offset to the current position minus the anchor position and continues 
to step SI 98. In step SI 98, the control routine determines whether more starting or 
ending positions exist in the current annotation. If, in step SI 98, the control routine 

1 5 determines that more starting or ending positions exist in the current annotation then 
the control routine returns to step SI 86. If, on the other hand, the control routine 
determines, in step SI 98, that no more starting or ending positions exist in the current 
annotation then the control routine continues to step S200. 

In step S200, the control routine initiates a loop to process all strokes in the 

20 current annotation. In step S200, the control routine creates an anchored stroke 

having a composite anchor based upon the anchors corresponding to the starting and 
ending positions and continues to step S202. In step S202, the control routine sets an 
offset to the current position in the current stroke minus the starting position and 
continues to step S204. In step S204, the control routine divides the offset by the 

25 distance between the starting and ending positions and continues to step S206. In step 
S206, the control routine rotates the offset around an origin by an angle equal to the 
ending position minus the starting position and continues to step S208. In step S208, 
the control routine appends the offset to the anchored stroke's list of offsets and 
continues to step S210. In step S210, the control routine determines whether more 

30 positions exists in the current stroke. If, in step S210, the control routine determines 
that more positions exists in the current stroke then the control routine returns to step 
S202. If, on the other hand, the control routine determines, in step S210, that no more 
positions exists in the current stroke then the control routine continues to step S212. 



In step S2 12, the control routine adds the composite anchor to the annotation database 

and continues to step S214. In step S214, the control routine determines if more 

strokes exists in the current annotation. If, in step S214, the control routine 

determines that more strokes exists in the current annotation then the control routine 

5 returns to step S200. If, on the other hand, the control routine determines that no 

more strokes exists in the current annotation, then the control routine continues to step 

S216 where the control routine returns to step SI 12. 

Figs. 5A-5B shows a control routine for rendering freeform digital ink 

annotations on a changed view. The flow chart parallels the following pseudocode. 

10 Using the annotation database, get the list of all anchored strokes 

For each anchored stroke, 

Extract the anchor and the list of offsets 
Let stroke be a new stroke 
If anchor is a simple anchor 
1 5 Determine if the character or character position underlying the 

anchor has been deleted 
If not, then 

Let anchor position be the position of that anchor 
For each offset, 

20 Let new position be offset + anchor position 

Add new position to the stroke 
End for 

End if 

Else 

25 Determine if the character or character positions underlying the 

1 st and 2 nd anchors has been deleted 
If both have been deleted, then 
do nothing 

Else 

30 Let starting position be the position of the 1 st anchor, 

plus the 1 st offset 

Let ending position be the position of the 2 nd anchor, 
plus the 2 nd offset 
For each offset, 

35 Let new offset be offset rotated byangle (ending 

position minus starting position) 
Multiply new offset by the distance between the 
starting and ending positions 
Let new position be new offset plus starting 

40 position 

Add new position to the stroke 
End for 

End if 

End if 

45 Render stroke on the view 



End for 

The control routine starts at step S300 and continues to step S302. In step 
S302, the control routine acquires a list of all anchored strokes from the annotation 
database and continues to step S304. In step S304, the control routine extracts the 
5 anchors and the list of offsets from the current anchored stroke and continues to step 
S306. In step S306, the control routine lets the stroke be a new stroke and continues 
to step S308. In step S308, the control routine determines whether the anchor is a 
simple anchor. If, in step S308, the control routine determines that the anchor is a 
simple anchor then the control routine continues to step S3 10. If, on the other hand, 

10 the control routine determines, in step S308, that the anchor is not a simple anchor 
then the control routine jumps to step S326. In step S3 10, the control routine 
determines if the character or character position has been deleted. If, in step S3 10, the 
control routine determines that the character or character position has been deleted 
then the control routine jumps to step S320. If, on the other hand, the control routine 

1 5 determines that the character or the character position has not been deleted then the 
control routine then continues to step S3 12. In step S3 12, the control routine sets the 
anchor position to the position of that anchor for the offset and continues to step S3 14. 
In step S3 14, the control routine lets the new position be the offset from the stroke 
plus the anchor position and continues to step S3 16. In step S3 16, the control routine 

20 adds the new position to the stroke and continues to step S3 18. In step S3 1 8, the 
control routine determines whether there are more offsets for the anchor. If, in step 
S3 18, the control routine determines that there are more offsets then the control 
routine returns to step S3 14. If, on the other hand, the control routine determines that 
there are no more offsets then the control routine jumps to step S320. In step S320, 

25 the control routine renders the stroke on the view and continues to step S322. In step 
S322, the control routine determines if there are more anchored strokes in the 
annotation database. If, in step S322, the control routine determines that there are 
more anchored strokes in the annotation database then the control routine returns to 
step S304. If, on the other hand, the control routine, determines in step S322, that 

30 there are no more anchored strokes in the annotation database then the control routine 
continues to step S324 where the control routine returns control to the environment in 
which the control routine operates. 



In step S326, the control routine determines if the character or character 
position underlying the first and second anchors been deleted. If, in step S326, the 
control routine determines that both of these character or character positions have 
been deleted then the control routine returns to step S320. If, on the other hand, the 
5 control routine determines, in step S326, that the character or character position 

underlining the first and second anchors have not been deleted then the control routine 
continues to step S328. In step S328, the control routine sets the starting position to 
the position of the first anchor plus the first offset and continues to step S330. In step 
S330, the control routine sets the ending position to the position of the second anchor 

1 0 plus the second offset and continues to step S332. In step S332, the control routine 
sets the new offset to the offset from the anchored stroke rotated by an angle equal to 
ending position minus starting position and the control routine continues to step S334. 
In step S334, the control routine multiplies the new offset by the distance between the 
starting and ending positions and continues to step S336. In step S336, the control 

1 5 routine sets the new position to the new offset plus the starting position and continues 
to step S338. In step S338, the control routine adds the new position to the stroke and 
continues to step S340. In step S340, the control routine determines whether more 
offsets exist. If, in step S340, the control routine determines that more offsets exist 
then the control routine returns to step S332. If, on the other hand, the control routine 

20 determines, in step S340, that no more offsets exists then the control routine returns to 
step S320. 

Freeform digital ink annotation is intended to mean a visible ink stroke of 
arbitrary shape made with a pen, a mouse or other pointing device that persists over 
time. A pen based input device is intended to include a pen tablet, a pen display, a 
25 pen display tablet, a pen computer and/or a touch screen. The systems and methods 
of this invention may infer some meaning from the shapes or the morphological 
characteristics of ink strokes but do not remove the ink stroke once they have been 
recognized. 

While the detailed description refers generally to electronic systems, it is 
30 intended to be understood that the method and system of the present invention may be 
applied to any electronic book and document reader that displays changing views. 
Electronic book and document readers include any electronic handheld device that can 
be accessed with a pointing device such as a pen and that accepts freeform digital ink 



as input. Electronic book and document readers may also include other additional 
features without limitation. 

It is intended to be understood that the term "text" is intended to include text, 
digital ink, audio, video or any other content of a document, including the document's 
structure. An object is intended to be any component of a display such as, by way of 
non-limiting example, a character, a word, a phrase, a sentence, a paragraph, a line, a 
section, a chapter, a portion of an image that is identifiable by a computer vision 
system in a single image or in a stream of images as in a video, etc. 

It should be understood that a change in the structure of a document is 
intended to include inserting, deleting, copying and moving of objects or the like. It is 
also intended to be understood that a change in the layout structure of a document 
includes reflowing of objects, resizing of objects, rotation of objects, stretching of 
objects, distortion of objects, morphing of objects or the like. 

While this invention has been described in conjunction with the specific 
embodiments outlined above, it is evident that many alternatives, modifications and 
variations are apparent to those skilled in the art. Accordingly, the preferred 
embodiments of the invention as set forth above are intended to be illustrated and not 
limiting. Various changes may be made without departing from the spirit and scope 
of the invention. 



WHAT IS CLAIMED IS : 

1 . A method for maintaining an indication of a logical relationship of at 
least one freeform digital ink annotation to at least one object in a view as the view 
changes, the indication being a layout relationship of the at least one freeform digital 
5 ink annotation to the at least one object, the method comprising: 

mapping the logical relationship of the at least one freeform digital ink 
annotation to the at least one object based upon the layout relationship of the at least 
one freeform digital ink annotation and the at least one object in the first view; 
generating a second view; 
10 rendering the at least one freeform digital ink annotation in response to 

the generation of the second view, the layout relationship of the at least one freeform 
digital ink annotation and the at least one object in the second view based upon the 
mapped logical relationship of the at least one freeform digital ink annotation to the at 
least one object in the first view. 
15 2. The method of claim 1 , the at least one object being in an electronic 

book and document reader. 

3. The method of claim 1, the at least one object being in a word 
processor. 

4. The method of claim 1, the second view being generated in response to 
20 a change in the structure of a document. 

5. The method of claim 1, the second view being generated in response to 
a change in the layout of a document. 

6. The method of claim 1, the step of mapping comprising creating an 
anchor for the at least one freeform digital ink annotation. 

25 7. The method of claim 1, the step of mapping comprising: 

splitting the at least one freeform digital ink annotation into a plurality 
of substrokes, each substroke corresponding to the portion of the annotation that is in 
proximity to an object; and 

creating an anchor for each of the plurality of substrokes based upon 
30 the layout relationship of each of the plurality of substrokes to at least one object in 
the first view. 

8. The method of claim 7, further comprising generating connecting lines 
between substrokes. 



9. The method of claim 7, wherein the at least one object is split into at 
least two objects, wherein the rendering is in response to a separation of the at least 
two objects in the second view and wherein the at least one freeform digital ink 
annotation is split into at least two freeform digital ink annotations each 

5 corresponding to at least one of the at least two objects. 

10. The method of claim 1, wherein the at least one object comprises at 
least two objects and wherein the second view is in response to a merging of the at 
least two objects. 

1 1 . The method of claim 1 , wherein the second view is in response to a 
10 replacement of the at least one object with at least one new object. 

12. The method of claim 1 , the mapping depending upon the shape of the 
at least one freeform digital ink annotation and the proximity of the at least one 
freeform digital ink annotation to the object. 

13. The method of claim 12, the at least one freeform digital ink annotation 
1 5 being one of an underline and a cross-out annotation. 

14. The method of claim 12, the at least one freeform digital ink annotation 
being an insertion annotation. 

15. The method of claim 12, the at least one freeform digital ink annotation 
being a connector annotation. 

20 16. The method of claim 1 , the mapping depending upon the location of 

the at least one freeform digital ink annotation in the view. 

17. The method of claim 16, the location being in a margin of the first 

view. 

1 8. The method of claim 1 , further comprising displaying the first view in 
25 response to a selection of the at least one freeform digital ink annotation in the second 

view. . 

1 9. The method of claim 1 , the at least one freeform digital ink annotation 
including at least two freeform digital ink annotations and the at least one object 
including at least two objects that each correspond to one of the at least two freeform 

30 digital ink annotations, the method further comprising merging the at least two 

freeform digital ink annotations into a single freeform digital ink annotation when the 
corresponding at least two objects merge. 



20. A system for maintaining an indication of a logical relationship of at 
least one freeform digital ink annotation to at least one object in a view as the view 
changes, the indication being a layout relationship of the at least one freeform digital 
ink annotation to the at least one object, the system comprising: 
5 a display that displays the layout relationship of the at least one 

freeform digital ink annotation to the at least one object; and 

a processor that maps the at least one freeform digital ink annotation to 
the at least one object based upon the layout relationship of the at least one freeform 
digital ink annotation to the at least one object in a first view, the display being 

10 responsive to a change in the first view to generate a second view and to render the at 
least one freeform digital ink annotation in the second view, the layout relationship of 
the at least one freeform digital ink annotation and the at least one object in the second 
view being based upon the mapped logical relationship of the at least one freeform 
digital ink annotation to the at least one object in the first view. 

15 21 . The system of claim 20, the system being a word processor. 

22. The system of claim 20, the system being an electronic book and 
document reader. 

23. The system of claim 20, further comprising a housing that holds the 
display and the processor, the housing being of a size that is small enough so that the 

20 system is capable of being held by a hand, 

24. The system of claim 23, further comprising a pointing device. 

25. The system of claim 20, the change in the first view being a change in 
the structure of a document. 

26. The system of claim 20, the change in the first view being a change in 
25 the layout of a document. 

27. The system of claim 20, the processor mapping the at least one 
freeform digital ink annotation by creating an anchor for the at least one freeform 
digital ink annotation. 

28. The system of claim 20, the processor mapping the at least one 

30 freeform digital ink by splitting the at least one freeform digital ink annotation into a 
plurality of substrokes, each substroke corresponding to the portion of the annotation 
that is in proximity to an object; and 



creating an anchor for each of the plurality of substrokes based upon 
the layout relationship of each of the plurality of substrokes to at least one object in 
the first view. 

29. The system of claim 28, the processor also capable of generating 
5 connecting lines between substrokes. 

30. The system of claim 28, wherein the second view is generated in 
response to a splitting of the at least one object into at least two objects and to a 
separation of the at least two objects and wherein the at least one freeform digital ink 
annotation is split into at least two freeform digital ink annotations each 

10 corresponding to at least one of the at least two objects. 

3 1 . The system of claim 20, wherein the second view is generated in 
response to a replacement of the at least one object with at least one new object. 

32. The system of claim 20, wherein the at least one object comprises at 
least two objects and wherein the second view is in response to a merging of the at 

1 5 least two objects. 

33. The system of claim 20, the processor mapping depending upon the 
shape of the at least one freeform digital ink annotation and the proximity of the at 
least one freeform digital ink annotation to the at least one object. 

34. The system of claim 33, the at least one freeform digital ink annotation 
20 being one of an underline and a cross-out annotation. 

35. The system of claim 33, the at least one freeform digital ink annotation 
being an insertion annotation. 

36. The system of claim 33, the at least one freeform digital ink annotation 
being a connector annotation. 

25 37. The system of claim 20, the processor mapping depending upon the 

location of the at least one freeform digital ink annotation in the view. 

38. The system of claim 37, the location being in a margin of the first 

view. 

39. The system of claim 20, the at least one freeform digital ink annotation 
30 including at least two freeform digital ink annotations and the at least one object 

including at least two objects that each correspond to one of the at least two freeform 
digital ink annotations, the processor being responsive to a merger of the at least two 



objects to merge the at least two freeform digital ink annotations into a single 
freeform digital ink annotation. 

40. The system of claim 20, the display being further responsive to a 
selection of the at least one freeform digital ink annotation in the second view to 
display the first view. 



ABSTRACT OF THE DISCLOSURE 
The method and system of the invention maintains the indication of the logical 
relationship of freeform digital ink annotations to objects in an electronic display even 
as the view changes. The invention maps the logical relationship of the freeform 
5 digital ink annotations to objects in the view based upon the layout relationships of 
the freeform digital ink annotations to the objects in the view. If the view changes, 
the invention maintains the layout relationship of the freeform digital ink annotations 
to the objects in the view based upon the mapping of the freeform digital ink 
annotations to the objects. 
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From a cognitive perspective, writing involve* three key sub-processes: planning, drafting, and 
editing This model is not linear: sub-processes are intertwined and may be recursive 
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reading tources (includes annotation, note-taking, and i 




generating ideas (ie. free associating on a pad of paper: reviewing notes may fit here) 
re-orgaauang (of notes from so wees* new idea notes, etc. — includes outlining and lets 
structured actions like emphasising useful notes, making connections between them. etc. 
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section according to a new plan if the old version is discarded 
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reading sources ti eo much fun to do (includes annotation, note-taking, and 

(ie. firee associating on a pad of paper, reviewing notes may fit here/ 

w „ . . _ g. keep it simple for this audience: used more during revising or ) 
«^6nf5SffVprocesj of generating sentences according to apian. Drafting does not include 
/ making local changes, which is considered part of editing. Drafting does include rewriting a 
section according to a new plaa if the old version is discarded 
Editing or reviewing and revising includes the reviewfrevise cycle and local editing 
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401. 

The Problems 



<f|0 fa this sectioa w* ***** agg^ of problems that stand inthe way ° f ^ 

\ IbTem is descnW^alS^ * ,nvest. 8 aaon malmay^ 

tlead to a solution are pr escniwi in subsg gj™^ A 

. v 4o4- 

|q3 — . — J-f— prertrfiao /nfenrntfo>>~access on mootfe devfeaw « n«ro* 



Specific example s of mis pr o bl e m E<: 
^ . Not being able to get me infonnatkx. you need when you're away from your desktop. 
*J0(tf . Waitin g for Web pages to downlc>adov«^ 

(w of Dortable computers is infbnnation access anytime, anywhere. People have 
b^^S on c^SKgLpeed, lowest network, at their -^JP^* 
coders are rarely connected to such networks. Cactog a a techmque that m^ portable 
computOTle«dep<gidgntonubkiuito«alow^rt networking. *H4- 

~~ ^^n^anlncient computer science tecluikrue^ stores a copy of gentry -referenced 

SS^^JattanAASugh it is a simple concept, a number of subtle problems anse m 
^^^SortSpart of a «ching subsystem is the replacement policy: winch .terns should 
ST^vcd wShTlS storefi^? Tto isan inherently difficult decisis because rt 
n^^d^^Sm^or memos, part, the performance of a cache is determined by me 
cTca^if^Lement policy. A good policy has a large ratio of -hits" (.terns found » the 
cache) over "misses" (items found to be missing from the cache). 

CaMm has additional implications for mobile devices. When connecting over wireless _ 
W ^KSmisses^ionly take a relatively long drne to process, ^J%W™*£ 
^.o l^rve fees for network access. Moreover, when operating disconnected from a network, 
Jo^r- cache misses may stop users from continuing wimuWr work. 

2.1.1 Renroving dc<x«T»ents from 

Researchers have been mvestigating new replacement policies that brterpredict J*« 
Ser«*eTone avenue is to augment «mrputanonal predichons wm J^^f 
a^assunutioa mat people often know wtakh irfonnatiM is ni« 
^^SSoTand Mowgli [LUjeberg95] mobile web browsers aUow users to lock 
S^Z^SLtc. In Tele Web the cache contents axeexposed tothe user » ^« * 
Sti^usenn-ylc^ilen^ 
!3user-levdl«^ interface is 

rf^sSnci^approachistoasktheuw 
/ kS^Ke^ 
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